Mobile sessions

Now we have fixed sessions and mobile sessions available. Ideally, we will use mobile sessions to do any analysis, and use fixed sessions as “knots” to validate the measurements. However, we don’t know any models that can do that job. Anyway, I will pull data from mobile sessions first and see what they look like. I will also remove those that are obviously wrong.

I will follow Chris’s method:

• Step 1: find usernames from AirCasting website under the MAPS session
• Step 2: plug in usernames in the mobile session API call and get the IDs
• Step 3: plug in IDs in the measurement ID API call and get the measurements in the form of points

usernames <- c('NYCEJA', 'BLU%2012', 'HabitatMap', 'BCCHE', 'scooby', 'Ana%20BCCHE', 'Ricardo%20Esparza', 'Tasha%20kk', 'lana', 'Marisa%20Sobel', 'Wadesworld18', 'El%20Puente%201', 'El%20Puente%202', 'El%20Puente%203', 'El%20Puente%204', 'El%20Puente%205', 'mahdin', 'Asemple', 'patqc', 'sjuma')

user_test <- c('NYCEJA', 'HabitatMap', 'BCCHE', 'lana', 'Wadesworld18', 'patqc')

fetch_id <- function(name){
  api_call <- str_c('http://aircasting.org/api/sessions.json?page=0&page_size=500&q[measurements]=true&q[time_from]=0&q[time_to]=2552648500&q[usernames]=', name)
  api_pull <- jsonlite::fromJSON(api_call)
  user_id <- api_pull$streams$'AirBeam2-PM2.5'$id %>% 
    .[!is.na(.)]

  user_id
}

pulled_ids <- map(usernames, fetch_id) %>% 
  unlist()

# This function plug each ID into the measurement API call and pulls data using that ID
pull_fun <- function(id_element){
  test_sess <- str_c("http://aircasting.org/api/realtime/stream_measurements.json/?end_date=2281550369000&start_date=0&stream_ids[]=",id_element) %>% 
    jsonlite::fromJSON(.) %>% 
    mutate(id = id_element) %>% 
    as_tibble()

  test_sess
}

# the output of pull_fun is a list. Take each element of the list and combine them into a tibble
airbeam_data <- map(pulled_ids, pull_fun) %>% 
  do.call("bind_rows", .)

Data cleaning for mobile sessions

We did the following to clean the data:

1. Seperated date and time and created a date variable
2. Removed the values that beyond the range (0, 250), and divided the data into two subsets: a subset with regular values and the other with extremely high values. We analyzed them seperately
3. Removed or edited the latitude and longitude that were not in New York.

airbeam_data_tidy = airbeam_data %>% 
  separate(time, into = c("year", "month", "day"), sep = "-") %>% 
  separate(day, into = c("day", "time"), sep = "T") %>% 
  separate(time, into = c("hour", "min", "sec"), sep = ":") %>% 
  separate(sec, into = c("sec", "remove"), sep = "Z") %>% 
  select(-remove) %>% 
  mutate(
    date = str_c(year, month, day, sep = '-'),
    date = as.Date(date)
  ) %>% 
  filter(value > 0 & value < 1000) %>% 
  filter(latitude > 38 & longitude < -70) %>% 
  filter(latitude > 40 & longitude > -75)

Subset 1: regular measurements

airbeam_reg <- airbeam_data_tidy %>% 
  filter(value <= 250)

Subset 2: extremely high values

airbeam_high <- airbeam_data_tidy %>% 
  filter(value > 250)

Our analysis willl primarily focus on the regular measurements. However, extremely high measurements are also useful, because they could help identify potential sources that cause peaks in PM2.5. These two subsets will be analyzed seperately.

Visualization

Here are the questions that we want to answer in the visualization:

1. In general, where are PM2.5 measured in the city? Which areas have more observations and which has less?
   
2. How does PM_2.5 observations change throughout a day?
3. Are there any monthly or seasonal trends?
4. Where do the extreme values appear?

Q1: Where are PM2.5 measured?

Here’s a plot that shows the spatial distribution of all the locations. I have already removed those that are not in New York.

location_reg <- airbeam_reg %>% 
  group_by(latitude, longitude) %>% 
  summarize(avg_pm = mean(value)) 

leaflet() %>% 
  addTiles() %>% 
  addCircleMarkers(
    data = location_reg,
    lat = ~latitude, lng = ~longitude,
    color = 'green',
    radius = 3
  )

Q2: How does PM_2.5 observations change throughout a day?

airbeam_reg %>% 
  mutate(hour = as.numeric(hour)) %>% 
  group_by(hour) %>% 
  summarize(pm_avg = mean(value)) %>% 
  ggplot(aes(x = hour, y = pm_avg)) + geom_line()

Q3: Monthly or seasonal trends?

monthly_averages = airbeam_reg %>% 
  group_by(month, year) %>% 
  mutate(average = mean(value))

airbeam_reg %>%
  group_by(month, year) %>%
  summarize(average = mean(value), pm_max = max(value), pm_min = min(value), observation_count = n()) %>%
  knitr::kable()

In this table, we see that average PM2.5 peaks in the summertime.

Q4: Where do the extreme values appear?

location_high <- airbeam_high %>% 
  group_by(latitude, longitude) %>% 
  summarize(avg_pm = mean(value)) 

leaflet() %>% 
  addTiles() %>% 
  addCircleMarkers(
    data = location_high,
    lat = ~latitude, lng = ~longitude,
    color = 'green',
    radius = 3
  )